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Achieving success in information system and technology (IS/IT) projects is a 
complex and multifaceted endeavour that has proven difficult. The literature 
is replete with project failures, but identifying the critical success factors 
contributing to favourable outcomes remains challenging. The triad of Time- 
Cost-Quality is widely accepted as key to achieving project success. While 


time and cost can be quantified and measured, quality is a more complex 
construct that requires different metrics and measurement approaches. 
Utilizing the PRISMA Methodology, this study initiated a comprehensive 
search across literature databases and identified 142 relevant articles 
pertaining to the specified keywords. A subset of ten articles was deemed 
suitable for further examination through rigorous screening and eligibility 
Success factor assessments. Notably, a primary finding indicates that despite recognizing 
Systematic literature review usability as a critical element, there is a tendency to neglect usability 
Usability enhancements due to time and resource constraints. Regarding the influence 
of usability on project success, the active involvement of end-users emerges 
as a pivotal factor. Moreover, fostering the enhancement of Human- 
Computer Interaction (HCI) knowledge within the development team is 
essential. Failure to provide good usability can lead to project failure, 
undermining user satisfaction and adoption of the technology. 
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1. INTRODUCTION 

Extensive research has been conducted on information technology (IT) projects’ success and failure 
factors, revealing the inherent risk involved in such endeavors [1]-[3]. It is well established that achieving 
success in IT projects is a complex and challenging process that requires careful planning, execution, and 
management. Despite the significant efforts invested in IT project management, the failure rates still need to 
be lowered. Therefore, understanding the critical factors that impact project success, such as usability, is 
essential for achieving favorable outcomes. 

However, it is important to note that successful information system and technology (IS/IT) projects 
are not a rare phenomenon [4], [5]. While failures in IS/IT projects have been widely studied, critical success 
factors that contribute to favorable outcomes have also been extensively researched. It is imperative to 
identify and prioritize these success factors to mitigate risks and ensure the success of IS/IT projects. 

Within the triad of cost, time, and quality, quantifying quality is often the most challenging, as it 
encompasses a range of factors that vary across projects. While cost and time can be more easily measured, 
the measurement of quality is multifaceted and can include aspects such as maintainability and usability. The 
present study will focus on usability as a critical quality factor in the success of IS/IT projects. Despite being 
an integral aspect of computing since its inception, the contribution of usability to project success has yet to 
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be fully explored. This paper aims to shed light on this important but often overlooked aspect of IT project 
management. 

The following research questions will be answered throughout this systematic literature review: 

— Howis Usability positioned in IS/IT Project? 
— How to improve the quality of usability factor in the IT Project Success? 

This paper is structured as follows: section 1 provides an introduction to the topic, while section 2 
discusses the background and related works (meta-studies) in the field of usability and IS/IT projects. In 
section 3, the research questions and methodology will be presented in detail. Sections 4 through 6 will cover 
the results for each of the research questions. Finally, section 7 will conclude the paper 


2. LITERATURE REVIEW 

This section will explore literature relevant to the management of IS/IT projects concerning 
usability. We will also examine how usability is an essential aspect of system acceptability. Furthermore, we 
will review literature focusing on the success criteria for IS/IT projects, including project management 
success, system quality, and project product success. 


2.1. Background 

Firstly, research about usability and usability evaluation, including the user experience evaluation, is 
easily found in academic databases. The methodology of usability evaluation also developed into many kinds 
of user interaction, such as e-government [6], mobile application [7], chatbot [8], and virtual reality [9]. In 
addition to that, Systematic Literature Review about the usability and user experience also is a common 
article published in computer-science-related journals and publications. 

On the other hand, IS/IT projects have also been the subject of extensive research, which includes 
investigations into success factors, failure rates, and systematic literature reviews. Such research is readily 
accessible, and the findings can be leveraged to enhance project management practices. Although the 
researchers have not yet come to one conclusion about what is the most critical success or failure factors, it’s 
worth noting that likely most parts of the IT project have been researched. 

However, the research about the roles of usability in IS/IT projects has yet to be researched to a 
considerable extent. This is an important area of research, as usability is a key factor in the success of any 
IS/IT project. This research aims to fill this gap by reviewing the literature on usability and IS/IT projects 
from the past decade and synthesizing the main findings. 


2.2. Usability 

Usability is defined in ISO/IEC 25010 as “the degree to which a product or system can be used by 
specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified 
context of use” [10], [11]. While another definition describes usability as being “easy to learn, useful 
(provide necessary functionality), easy to use and pleasant to use” [12]—[14]. Despite the various definitions, 
usability is widely acknowledged as being context-dependent. Another scholar emphasizes that usability 
needs to be referenced to a particular context [15]. 

Usability is a component of user experience (UX), which is part of practical acceptability according 
to Nielsen’s model [16]. The components of usability are shown in Figure 1. The figure also shows the 
position of usability related to the system and practical acceptability. 

Measuring usability presents challenges due to its multidimensional nature, influenced by various 
factors such as the user’s task, the product itself, and the specific environment [17], [18]. To address this, 
researchers often adopt an approach that involves engaging participants to perform tasks using the system, 
and then assessing key factors like the time taken to complete tasks, the rate of successful scenario 
completions, and the time required for error recovery [19]. This task-oriented evaluation allows for a more 
comprehensive understanding of usability and its impact on the overall user experience, making it a valuable 
method in usability research. 

In the context of project management, usability is considered a non-functional requirement (NFR) 
and part of the quality factor [20], which is one of the three common criteria used to measure project success 
[21], [22]. Previous research has demonstrated that usability can improve satisfaction, productivity, and 
technology acceptance, ultimately contributing to the success of IS/IT projects [8]. Therefore, understanding 
and prioritizing usability as a critical quality factor is essential for achieving favorable outcomes in IT 
projects. 

To strengthen the argument for how usability success contributes to the overall success of an IS/IT 
project, prior research has demonstrated that improved usability leads to enhanced user satisfaction, increased 
productivity, and better technology acceptance [9]. Furthermore, assessing usability poses unique challenges 


Comput Sci Inf Technol, Vol. 5, No. 1, March 2024: 7-12 


Comput Sci Inf Technol ISSN: 2722-3221 o 3 


due to its multidimensional nature, which is influenced by various factors specific to the user’s task, the 
product, and the environment in which it is used, making it more complex than traditional one-dimensional 
product or user characteristics [10], [11]. This nuanced understanding of usability’s impact on the user 
experience underscores its crucial role in the success of IS/IT projects and highlights the need for tailored 
approaches in evaluating and enhancing usability. 

In conclusion, usability is a critical quality factor in IS/IT projects that can impact project success. 
Usability is challenging to measure due to its multidimensional nature and its dependence on context. 
However, assessing usability can be achieved by asking participants to perform tasks using the system and 
measuring factors such as task completion time, success rate, and error recovery time. Prioritizing usability as 
an NFR can improve satisfaction, productivity, and technology acceptance, ultimately contributing to the 
success of IS/IT projects. 


Social Acceptability 


Utility Subjectively Pleasing 
Usefulness 
System Acceptability Easy to remember 
Easy to leam 


Usability 
Efficient to use 


Practical Acceptability FewE 
Few Errors 
Cost 


Compability 
Realibility 


Etc. 


Figure 1. Model of the system acceptability attributes by nielsen (recreated) 


2.3. IS/AIT project 

A project is an endeavor undertaken to create a unique product, service, or other result [23]. This 
endeavor has to be temporary. Hence, we could easily distinguish the project from operational activities. 
IS/IT Project is not defined far from the project definition as well. Bannerman presents a definition for 
“IT Project”, “Discrete and unique activities that serve as vehicles of multidimensional IT-based change” 
[24]-[26]. 

The word “multidimensional” serves well in today’s dynamics of business and other sectors of the 
world. Businesses are not only the ones who benefit from IS/IT development but every other organization, 
from the government to the private sector, from small organizations to large enterprises. Nowadays, IT 
Projects become a vehicle of transformation and growth for the business. The global IT Investment in 2022 
was estimated by Gartner to grow and reaches 4,4 trillion dollars. It only grows from previous years amid the 
pandemic. 

Nevertheless, there is complexity in every IS/IT Project that has been kicked off. This is the one that 
makes an IS/AIT Project unique. Moreover, one project had a different complexity from another, and this not 
only makes it unique but also makes it prone to failure [3], [27]. But as it was mentioned before, the failure 
and success of IT Projects have been widely researched in empirical studies and systematic reviews. This 
paper is not research on IT Projects and failure but is more focused on the usability in IS/IT Projects. 

Although IT project success is important for the overall IT success [28], there are no formal criteria 
for project success. However there is a criterion that is stated very well by Iriarte and Bayona [25]. The 
criteria are divided into two groups which are project management success and project product success. Each 
group has three criteria, and there are system quality criteria that fall into both groups. The list of project 
success criteria is shown in Figure 2. 

A systematic literature review comprising 39 studies conducted by Iriarte and Bayona identified 
several critical success factors in IS/IT projects [25]. These factors include the capability and skills of team 
members, user involvement, top management support, consultant knowledge and technical expertise, 
organizational maturity and culture, internal communication, project organization structure and 
responsibilities, effective planning, and the ability to handle changes and deviations. Lamprou identified 
additional critical success factors, including project finance/funding and economics, team competency and 
effectiveness, and project managers’ capability and experience [5]. 

However, research has also shown that inadequate attention to these critical success factors can lead 
to project failure. Inadequate management support, lack of technical know-how among team members, and 
lack of project managers’ capability and experience have been identified as factors contributing to project 
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failure. Other factors identified in the literature include poor supplier selection, data-related technical 
problems, and insufficient user involvement and commitment [29]. The causes of project failure can vary 
depending on the project lifecycle or phase, geography, and type of measured failure. 

Recent research has highlighted the need for customized approaches to understanding project failure 
in IS/IT projects. Gupta has suggested the importance of researching project failure prediction under dynamic 
scenarios [2], while Money has proposed a forensic analysis framework to comprehensively understand the 
causes of project failure in IS/IT projects [30]. These approaches are based on the principle that a 
comprehensive understanding of each project is necessary to understand why, where, when, how, and what 
causes project failure. Another research shows that IT project failure comes from the neglect of Non- 
Functional Requirements in the Development phase [31]. The findings of these studies emphasize the 
importance of identifying and addressing critical success factors and potential failure points to improve 
project outcomes in IS/IT projects. 

According to Figure 2, which outlines the criteria for project success, system quality is an essential 
component of both project management success and project product success. System quality is a measure of 
users’ perceptions of adaptability, availability, reliability, response time, and usability [32], [33]. Specific 
usability characteristics, such as appropriateness, recognizability, learnability, and operability, are also 
associated with improved system quality across four factors, including custom software quality, 
documentation, time and budget, and new package functionalities [33]. 

Given these findings, it is evident that usability is a crucial factor in achieving success in IS/IT 
projects. By improving system quality, usability can positively impact project management success and 
project product success. Therefore, it is crucial to prioritize usability as an essential component of IS/IT 
projects to improve the overall success of the project. 


Project Management Success 


Time and Budget Project Management 


Project Stakeholder 
On Time Scope / Specifications Goals Achievement ne - — 
Satisfaction 
em Quality of Project 
in Budget Process Efficiency | Management Team Satisfaction 


System Quality 


( System Quality } [ Information Quality ] 


User Satisfaction Economic Value 


User / Customer a, ; 
Satisfaction Use / Intention to Use Business Impact Impact on Users 


Project Product Success 


Figure 2. Project success criteria by iriarte (recreated) 


3. METHOD 

This paper employs the preferred reporting items for systematic reviews and meta-analyses 
(PRISMA) methodology to conduct a literature review. PRISMA is a widely-used methodology in the 
academic world for conducting systematic reviews or meta-analyses on a topic. The PRISMA methodology 
comprises four key steps: identification, screening, eligibility, and inclusion [34]. Table 1 outlines the 
inclusion and exclusion criteria employed in this study. The straightforward description of the method 
depicted in Figure 3. Using this methodology, this paper aims to provide a comprehensive review of the 
existing literature on usability and its impact on IS/IT project success. 


Table 1. Inclusion and exclusion criteria 


Inclusion Criteria Exclusion Criteria 
1. Either title, abstract, or keywords related to this paper’s topic 1. Paper in a language other than English 
2. Articles about IS/IT Projects and Usability 2. Unpublished articles 
3. Published in the range of ten years (2012 — 2022) 3. Duplicates articles 
4. Journal articles and conferences proceedings 4. Lecturer Note, review articles, reports 
5. Able to answer the RQ 5. Systematic literature review and Meta-analysis 


6. The whole paper is available for the authors 
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Records identified trough Additional records identified 
database searching through other sources 
(n= 142) (n= 2) 


i 


Records after duplicates removed 
(n=137) 


Records screened Records excluded 
(n=137) (n=126) 
Full-text articles assessed Full-text articles excluded, 
for eligibility «| with reasons 
(n=11) (n=1) 


—— 


Studies included in 
synthesis 
(n=10) 


Figure 3. PRISMA methodology flowchart 


3.1. Identification 

This PRISMA Method was applied to a literature review from five of the academic databases. These 
academic databases were chosen since they are both prominent and peer-reviewed. The academic databases 
are ACM Digital Library (https://dl.acm.org/), Emerald Insights (https://www.emerald.com/insight/), 
IEEExplore (https://ieeexplore.ieee.org/Xplore/home.jsp), ScienceDirect, and Scopus 
(https://www.scopus.com/search/form.uri). 

In searching for the related studies, these main keywords were used, “Information System Projects” 
AND “usability”. Next, the term “Information System Projects” expanded into similar words and terms such 
as Information Technology as well as the abbreviation. Thus, the produced keywords are as follows, “IT 
Project”, “IT Projects”, “Information Technology Project”, “Information Technology Projects”, “IS Project”, 
“TS Projects”, “Information System Project”, “Information System Projects”. On the other hand, the usability 
keyword is not modified or added with other related terms such as User Experience or User Interface. These 
two other keywords will skew the query results since it has different meaning and academic topics. 


3.2. Screening 

The screening process is an important step in the PRISMA methodology as it aims to remove 
duplicate results from various databases. In some cases, articles may be indexed in two or more databases, 
resulting in duplication. To identify and remove these duplicates, researchers must manually read and 
compare the title, abstract, and keywords of each article. This ensures that only unique articles are included in 
the study and eliminates any potential bias or error resulting from the inclusion of duplicate results. 

During the screening process, any literature that cannot be retrieved in full text is excluded. This 
step is necessary as it allows researchers to determine the quality, details, and methodology of the articles. 
Only literature that can be retrieved in full text can be properly evaluated and, therefore, cannot be included 
in the study. By excluding these articles, researchers can ensure that the study is based on a thorough and 
reliable analysis of the existing literature. 


3.3. Eligibility 

The third step in the PRISMA methodology is the eligibility check, which involves reading the 
entire article that passed the screening step. During this step, researchers evaluate the article based on the 
inclusion criteria, which typically involves assessing whether the article can answer at least one of the 
research questions. If an article meets the inclusion criteria, it will be selected and included as a reference for 
the research. By conducting a thorough eligibility check, researchers can ensure that the selected articles are 
relevant and reliable and can provide valuable insights into the topic being studied. 
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3.4. Included 

After passing the first three steps, the last step in PRISMA is to determine how the articles act as a 
source for this current systematic review. In the PRISMA model in Figure 3, there are two articles that are 
included from other sources, whilst both obtained from the same database source as the database searching. 
These two articles were selected from dl.acm.org and scopus.com since it has the same literature relation and 
connection to this current research. The final results of the method finally included 10 literatures, which 
shown in the Table 2 from identification process up to the included. 


Table 2. Comparison of number of researches by sources 


No. Source Identification Screening _ Eligibility _ Included 
1 ACM Digital Library 117 137 11 10 
2 Emerald 3 
3 IEEExplore 7 
4 — ScienceDirect 15 
5 Scopus 20 
Total 212 


4. RESULTS AND DISCUSSION 

In this section, the selected studies will be analyzed to facilitate comparisons between articles. The 
analysis will begin by examining the project management methodology, including whether the traditional 
waterfall or Agile methodology was used. The next focus will be the product developed in the project, with a 
brief analysis of the project results (whether it failed or succeeded) if applicable. However, the most critical 
aspect of the analysis will be the positioning of usability in the project, including whether the project 
management plan explicitly mentions usability as a key factor. Furthermore, Table 3, which contains the 
titles of the articles to be reviewed, will provide a comprehensive overview of the literature sources 
considered in this analysis. 

Given the crucial role of usability in IS/IT project success, it is essential to understand how it is 
positioned in each study. By examining how usability is incorporated into the project management plan, 
researchers can gain insights into the best practices for integrating usability into IS/IT projects. This analysis 
will contribute to a more comprehensive understanding of how usability impacts project success and provide 
valuable insights for future research in this area. 


Table 3. Included articles to be reviewed 


No. Published Year Article Title Reference: 

1 2013 The impact of end-user participation in IT projects on product 35] 
usability 

2 2014 Lean UX: The Next Generation of User-Centered Agile 36] 
Development? 

3 2016 Hard lessons learned: Delivering usability in IT projects 37] 

4 2018 Adapting Lightweight User-Centered Design with the Scrum-Based 38] 
Development Process 

5 2018 Configurations of user involvement and participation in relation to 39] 
information system project success 

6 2018 Delivering Usability in IT Products: Empirical Lessons from the 40] 
Field 

7 2019 Integrating HCI Perspective into a Mobile Software Development 41] 
Team: Strategies and Lessons from the Field 

8 2020 IT Project Failures, Causes and Cures [3] 

9 2021 Human Factors Methods Applied to a Healthcare Information 42] 
Technology Project 

10 2021 Information System Project Success Based on User Experience at 43] 


the Village Administration Office 


To simplify the presentation of the selected articles, they will be summarized in Table 4 based on 
the main problem of project management related to usability. The table will also include information on the 
type of research conducted to show how the research was carried out. Most of the selected articles are based 
on empirical research, with two articles focusing on specific case studies and one article using a qualitative 
comparative analysis approach. By presenting the information in a clear and concise format, the results of the 
literature review can be easily understood and used to inform future research in this area. 
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Table 4. Articles main problem 
No. Type of Research Project Management Problems Related to Usability 

1 Empirical Research Study Case with 15 IT Project managers about end-user 
participation in the final product usability. 

Problems identified: Involving user in IT Project would 
increase the cost, whilst PM need to undertake decision to 
minimize cost. 

2 Case Study Agile, which has gained popularity in the past decade, has 
yet to reach a similar de facto standard status as the old 
waterfall process for usability testing. 

Lean UX philosophy is identified by Design thinking, lean 
production, and Agile development. 

3 Empirical Research Most software vendors manage projects by encountering 
numerous issues in providing usability in IT solutions and 
successfully completing a project within a given time 
frame. 

Customers will perceive a low-end usability product as 
low quality as a whole. 

The risks associated with the failure to provide usability 
were assigned to the project manager or the entire 
company in charge of the project. 

4 Empirical research of two projects There are several things that could be improved in 
managing IT Projects in developed countries. 

The vendors who can meet the demand for IT projects 
lack skills and unrealized benefits, as well as an 
inadequate education and training support infrastructure 
and the absence of national guidelines and standards. 

The project owner failed to take explicit usability 
requirements into account in the call for tender documents 

BS) Qualitative Comparative analysis Even though it has been suggested that UIP is important 
in requirement analysis or during testing to influence 
project success positively, there is a lack of knowledge 
about the appropriate timing for UIP. 

Proposing how to involve users in an IS/IT Project 

6 Empirical Research It is not easy to reveal the factors that impact software 
product usability and the success of an entire project. 

The aim is to determine factors with a positive or negative 
impact on delivering usability in software products and 
the success or failure of an entire project. 

7 Case Study A small company does not have enough resources for 
HCI, whilst a software engineer is more concerned with 
the performance and capabilities of the developed 
software. This problem increases if the software 
development method is Agile. 

8 Empirical Research There are five failed projects that have been investigated, 
and one of the problems is usability which needs to be 
addressed. 

Need to perform the usability test better, e.g., only asked 
about the user interface with a few users. 

Did not check usability with real users because the 
assumption was that since the expert had designed it, it 
was good. 

9 Case Study The availability of patient demographic data and poor 
quality of clinical referrals. 

Healthcare quality improvement projects often use a 
narrow range of metrics that provide just one measure of 
systems. 

10 Case Study ISAT Implementation projects come into a fail because of 

a lot of factors, and User Experience is one of them. 
The quality system is the most influential factor in 
information system project success based on User 
Experience, while the quality of the system itself is 
influenced by the constructs of usability, affect, and user 
value. 


Once the main problems identified in each selected article have been analyzed, Table 5 will be used 
to summarize the research findings and recommendations. This summary will serve as a basis for the analysis 
related to the research questions in this paper. The column numbering of Table 5 corresponds to Table 3, 
which lists the articles to be reviewed. By presenting the research findings and recommendations in a concise 
and organized manner, this table will facilitate the comparison and synthesis of the results, providing insights 
into the best practices for incorporating usability into IS/IT projects. 
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Table 5. Research recommendations 


No. Findings Recommendations 

1 The study of user participation in IT projects Improving the success rate of an IS/IT Project (particularly in 
demonstrated its significance in ensuring developing an Information system for an organization), it is important to 
product usability. involve real end-user. 

Collected data shows that product usability 
increased in projects where end users were 
involved. 

The findings of an examination of the factors 
influencing (high) product usability. 

This analysis reveals that the following 
factors influenced the user participation 
variable: group size, project participation 
stage, and group size, with group size having 
the greatest impact on product usability. 

2 Involving users in the classic waterfall Since Lean UX attempts to incorporate user testing into the scrum sprint 
method does not suitable for the Agile or other Agile Timebox, it is important to involve users in carrying out 
method. regular testing of the deliverables or the increment of the sprint. 

3 The survey of projects also revealed that The author proposes a set of rules that could help the IT Project deliver 
formalization of project documentation _ better usability. 
ensures high usability. From the ten rules developed, rules number 1,2,3,4,7,9, and 10 contain 

direct guidelines to improve the quality of usability. For this paper, rules 
number 1 and number 2 will be summarized as "the better the 
formalized documentation, and the higher the involvement of the end- 
user, then the higher the usability quality.” 

4 There is a real problem regarding IT project There is a need to develop formal standards and guidelines for the 
usability, especially in developing countries. | government to improve the IT Investment of the government itself. In 
Lack of IT skill and experience, combined _ this case, these are standards and guidelines for usability or other quality 
with the organization’s failure to state the attributes for Information System Development. While the author 
quality attribute such as usability on the involves in the project as a Human-Computer Interaction researcher, 
procurement document, lead to failure in IT joined the project team, and made usability tests and recommendations, 
Project. the usability quality of the software was improved. 

5 User involvement is important, but only some During the user requirements analysis phase, project managers should 
phases of the project need the end user. This — ensure that they focus their attention on user participation. They should, 
paper concludes that the best phase for in particular, ensure that actual end-users of the IS participate. This 
involving the user is the requirement phase — should help to improve the software’s perceived usability. 
and the testing phase. 

6 There are several factors that contribute to the We should bring software developers and UI designers closer together to 
project’s failure, and usability is one of the ensure that a reported and approved change is both possible to 
factors. Usability issues have happened in all implement and does not degrade usability. 
types of project management methods, such We believe that software architecture should include not only the 
as waterfall or Agile. technical aspects required to develop and implement a product (namely 

functional requirements) but also usability interaction with users as part 
of the non-functional requirements. 

7 To verify the impact of changes on usability At the end of the case study, the end-users said that there is a better 
and user experience, we conducted an dialogue and understanding between clients and developers (IT 
interaction flow review. vendors). Moreover, the improvement is also measured in the design 
This analysis focused on user interface and quality, usability, and user experience. In addition to that, the feeling of 
interaction elements, including validation security and bravery in the development while there is an HCI expert 
elements such as multiple features, UI style, joining the development team. 
overall usability, and user ratings. 

8 IT Project failure is preventable, whether in The author develops a causes and cures model to map any kind of IT 
cost, time, and quality. One of the most Project fail factor to researched-based cures. Organizations could apply 
reported key fail factors is the failure to the cures throughout the project to avoid or mitigate the risks. 
predict the system integration and false 
estimation of the human factor (the 
developers and the end-users) 

9 The improvement of the healthcare IT Project The involvement of the HCI Experts in the hospital-based improvement 
could be reached through usability project has led to better product usability. The observed improvement is 
improvement of the Information System used. in the quality of clinical referrals and the availability of patient data, 

particularly demographic data. 

10 Inexperienced users in rural areas could be The most influential factor of IS/IT Projects based on User Experience is 
benefited from Information System in a better the quality system, and this quality is heavily influenced by Usability as 
way if the usability of the system is good. well as the effect and User Values. 

The authors developed a model that combines Government or other policymakers in rural areas regarding Information 
the User Experience model with Information technology should pay a better understanding and attention to the aspect 
System Project Success. of User Experience for every IS/IT Project kicked off. 


4.1. RQ1: How the usability positioned in IT project? 
The usability factor is already well-understood as a critical component of product quality in IS/IT 
project development. However, project managers often find themselves in a position where they must balance 
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the achievement of project goals in terms of time, cost, and quality. In other words, quality must compete 
with the other two factors, as achieving better quality often requires additional time and resources, including 
those needed to improve usability. 

Previous research suggests that some projects need to conduct usability tests on their products, 
despite recommendations to invest up to 10% of the budget in usability [44]. Unfortunately, this study was 
unable to determine whether this recommendation was applied in the projects reviewed. Moreover, it remains 
to be seen why many projects do not include usability requirements or allocate sufficient time and resources 
for usability testing and user involvement. 


4.2. RQ2: How to improve the quality of usability factor in the IT project success? 

The involvement of end-users in the project is widely recognized as a key factor in achieving project 
success, as stated in the literature review. However, the involvement must be done in the correct manner, as 
involving end-user managers is not suitable. End-user managers are not the actual end-users who will 
regularly use the system. Moreover, the involvement of end-users must be timed appropriately and taken into 
consideration during the construction phase. 

Furthermore, in addition to involving end-users, improving the development team's HCI knowledge 
or skill is also important. This can be achieved by providing additional resources to the team, such as hiring 
HCI experts or training software engineers in the basics of user experience. Such efforts will lead to the 
development of better products that meet end-users needs and expectations. 

In addition to project management and organizational efforts, it is also necessary for governments to 
develop and establish formalized standards and guidelines. These standards and guidelines can help small 
organizations that lack the resources for usability testing to improve the quality of their products. Ultimately, 
this can lead to the delivery of better products to clients and end-users. 


4.3. Discussion 

This subsection aims to contextualize the present study within the available body of evidence by 
providing a discussion that compares and contrasts the findings with those of similar studies. In doing so, we 
seek to elucidate the contribution of this research to the field of usability in IS/IT project management. 
Furthermore, the limitations of this study will be presented, along with suggestions for future research that 
could help fill the gaps in the current body of knowledge regarding usability in ISAT projects. 


4.2.1. Interpretation of results in the context of other available evidence 

The primary discovery of our study is that involving end-users and usability experts is crucial in 
enhancing the usability of the project’s end product. Improved usability implies better system quality, 
ultimately increasing the likelihood of success in ISAT projects. In addition to these findings, we have 
identified several other factors that should be considered by project members, such as formalized 
documentation, standards and guidelines, and communication and collaboration among designers, 
developers, and end-users. 

Our findings are consistent with a previous study conducted by Vince Bruno in 2011, which 
investigated the importance of usability in IS/IT projects. One of the primary outcomes of Bruno’s study was 
the significance of creating a “usability mindset” in key project stakeholders through a collaborative 
approach that highlights involvement in usability activities to attain a positive usability outcome [45]. In the 
research, he suggested creating a shared usability vision for the project or promoting a usability mindset that 
extends beyond the project and across the organizational culture to other projects. The concept of a usability 
vision is connected with usability standards and guidelines, as they foster shared understanding and 
expectation of the desired usability outcome of a project. Usability goals promote a usability mindset, 
extending beyond the project context mindset [46]. 

Four major themes emerged from Bruno’s research: usability mindset, collaborative approach, 
usability practice, and project constraints [45]. The most significant relationships found in the study are the 
two robust connections between the themes of a collaborative approach and a usability mindset. The research 
emphasized the importance of developing a symbiotic relationship between involvement and a usability 
mindset to achieve a beneficial usability outcome. Another noteworthy finding is the consistency of our 
results concerning the roles of usability in overall project success. Bruno’s research conducted in 2013 
reveals that usability is an iterative process that facilitates stakeholder involvement, articulates usability goals 
and requirements, and avoids technological constraints. These critical aspects are instrumental in achieving a 
successful usability outcome in a project [47]. 

Another study by Bruno proposed role differentiation between Usability consultants or experts and 
usability practitioners in IS/IT projects. The research revealed that there should be usability managers who 
focus on strategic usability issues and improving stakeholder collaboration, usability consultants who 
concentrate more on complying with usability activities within a process, and organizational practitioners 
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who focus more on nurturing and educating usability understanding within the organization and stakeholders 
involved in an IS/IT project. Furthermore, the study showed that an iterative usability process, stakeholder 
involvement, articulating usability goals and requirements, and avoiding technological constraints are critical 
to achieving a successful usability outcome in a project. 

While our research and the previous study both recommend the participation of end-users, usability 
experts, and practitioners, there are differences in emphasis. The previous research emphasizes the 
involvement of usability practitioners rather than end-users. In contrast, our selected research shows that five 
studies suggest end-user’ involvement and two studies suggest the involvement of usability experts. 

In alignment with the broader objectives identified within the realm of IT project management, 
recent studies corroborate the significance of user engagement and satisfaction in determining project success 
[48]. Husam AlFahl’s research, particularly relevant in this context, identifies three pivotal constructs — 
application design, application utility, and user interface structure — as the strongest predictors of user’s 
continued intention to utilize a mobile application and their loyalty towards it. This finding underscores a 
parallel in our research, reinforcing the notion that the sustained engagement of end users is a crucial, albeit 
often underemphasized, objective in the lifecycle of IT projects. Emphasizing these aspects can significantly 
contribute to the holistic success and long-term viability of IT initiatives. 


4.2.2. Limitation and future research 

The research in this paper has certain limitations, such as a relatively small number of studies 
included. To address this, it may be necessary to expand the search to include other databases and sources of 
information. Additionally, the research may not have been conducted in a thorough or rigorous manner, and 
the authors acknowledge that there may be more information to be gained from the selected previous research 
studies. In future research, it would be beneficial to extend the search for academic literature beyond the 
sources used in this paper. 


5. CONCLUSION 

In conclusion, this paper makes a substantial contribution by highlighting the crucial role of quality 
enhancement through usability in the realm of IS/IT projects. The success of an IS/IT project should not be 
measured solely in terms of time, financial resources, and technical excellence. Instead, it requires in-depth 
consideration of user experience. Even if a project is executed within its allocated resources and schedule, its 
long-term viability can still be compromised if end-users are not provided with a satisfying experience. This 
research proposes three strategic approaches for improving usability: user engagement, the involvement of 
HCI experts, and the adoption of formalized usability standards and guidelines. Organizations have the 
option of implementing incremental improvements in usability or can seek the expertise of HCI professionals 
for a more standardized approach. It is essential to understand that IS/IT projects involve significant 
investments and carry inherent risks of failure. Therefore, comprehensive measures are imperative to prevent 
yet another project failure. 
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